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Abstract 

To  examine  how  patient  safety  considerations  are  incorporated  into  medical 
device  purchasing  decisions,  individuals  involved  in  recent  infusion  pump 
purchasing  decisions  at  three  different  health  care  organizations  were  interviewed 
using  a  structured  interview  process.  Interview  questions  covered  triggers  for  the 
purchasing  process;  the  purchasing  process  itself;  how  safety  was  evaluated  and 
incorporated  into  the  process;  and  the  perceived  decision  and  process  quality.  The 
results  show  strengths  and  weaknesses  within  the  processes.  Strengths  included 
(1)  a  general  perception  that  patient  safety  was  important  and  played  a  role  in  the 
decision;  (2)  the  inclusion  of  a  wide  range  of  stakeholders  in  the  decisionmaking 
process;  and  (3)  the  use  of  device  user  feedback  as  a  component  of  the  device 
evaluation  process.  Weaknesses  included  (1)  safer  devices  may  have  been 
overlooked,  as  very  few  alternative  devices  were  considered;  (2)  two  important 
stakeholders,  device  users  and  patients,  did  not  participate  directly  in  the 
purchasing  decisions;  (3)  the  device  selected  for  purchase  often  was  determined 
before  the  evaluation  process  had  been  completed,  and  the  evaluation  process  then 
was  used  to  justify  the  selection  decision;  (4)  although  participants  felt  they  had 
considered  device  safety,  the  device  evaluation  often  was  limited  to  technical 
safety  issues,  such  as  operating  to  technical  specifications,  rather  than  device 
interface  issues  that  may  induce  or  prevent  device  usage  errors;  and  (5)  no 
explicit,  fonnal  usability  testing  was  conducted  at  any  of  the  three  sites  for  the 
purpose  of  assessing  device  safety.  These  weaknesses  underscore  the  need  for 
guidelines  and  tools  to  help  health  care  employees  better  assess  issues  central  to 
patient  safety  during  the  device  purchasing  process. 


Introduction 

Studies  show  that  more  problems  are  caused  by  medical  device  usage  errors, 
than  are  the  result  of  device  malfunctions.1  Although  device  users  are  traditionally 
blamed  for  these  errors,  there  is  widespread  evidence  to  indicate  that  a  large 
number  of  device  usage  errors  are  the  result  of  poorly  designed  user  interfaces.2 
The  United  States  Food  and  Drug  Administration  (FDA)  and  device 
manufacturers  continue  to  work  to  improve  device  safety,  but  since  no  device  is 
completely  safe  in  every  environment,  the  medical  device  purchaser  or  purchasing 
committee  is  responsible  for  selecting  the  medical  device  that  is  safest  for  their 
targeted  environment.  Unfortunately,  anecdotal  evidence  suggests  that  many 
health  care  professionals  believe  that  FDA  approval  ensures  device  safety  and  that 


341 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

MAY  2005 

2.  REPORT  TYPE 

N/A 

3.  DATES  COVERED 

4.  TITLE  AND  SUBTITLE 

5a.  CONTRACT  NUMBER 

The  Role  of  Patient  Safety  in  the  Device  Purchasing  Process 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Agency  for  Healthcare  Research  and  Quality  Office  of  Communications 
and  Knowledge  Transfer  540  Gaither  Road,  Suite  2000  Rockville,  MD 
20850 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR'S  ACRONYM(S) 

1 1 .  SPONSOR/MONITOR' S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release,  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

Published  in  Advances  in  Patient  Safety:  From  Research  to  Implementation.  Volumes  1-4,  AHRQ 
Publication  Nos.  050021  (1-4).  February  2005.  Agency  for  Healthcare  Research  and  Quality,  Rockville, 

MD.  http  ://www.ahrq.gov/qual/advances/. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

uu 

18.  NUMBER 
OF  PAGES 

12 

19a.  NAME  OF 
RESPONSIBLE  PERSON 

a.  REPORT 

unclassified 

b.  ABSTRACT 

unclassified 

c.  THIS  PAGE 

unclassified 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Advances  in  Patient  Safety:  Vol.  1 


any  approved  devices  must  be  comparably  safe.  Moreover,  it  is  unclear  to  what 
extent  the  health  care  professionals  charged  with  making  purchase  decisions  are 
trained  in  the  techniques  of  evaluating  patient  safety,  or  whether  these 
considerations  are  being  incorporated  into  the  device  purchasing  process.  It  is  also 
unclear  whether  those  making  the  purchase  choices  understand  how  to  make 
patient  safety  issues  a  priority  in  the  decisionmaking  process  and  what  criteria 
need  to  be  included. 

Despite  the  important  role  that  the  purchaser  plays  in  the  patient  safety  milieu, 
there  are  no  prior  studies  of  how  safety  considerations  are  integrated  into  the 
device  purchase  process.  Likewise,  there  are  very  few  published  guidelines  or 
procedures  for  assisting  the  purchaser  in  his  or  her  desire  to  make  patient  safety 
part  of  the  device  evaluation  process.  Although  the  FDA  clearly  recognizes  the 
importance  of  the  purchaser  and  the  end  user  in  decreasing  device  usage  errors, 
the  Agency  has  emphasized  guidelines  and  recommendations  for  medical  device 
manufacturers,  and  not  the  purchasers  of  the  equipment.3,4  The  FDA  guidelines 
for  purchasers  consist  of  a  one-page  list  of  questions  designed  to  assist  in  the 
selection  of  a  usable  device;  a  continuing  education  document  on  the  importance 
of  reporting  device  usage  errors;  a  quarterly  publication,  User  Facility  Reporting, 
that  addresses  usability  issues  from  time  to  time;  device  alerts  that  include  usage 
error  information;  and  an  online  form  for  reporting  device  usability  problems  that 
do  not  result  in  patient  harm.5, 6  And  while  a  very  few  academic  papers  have 
reported  on  safety  evaluation  procedures  for  specific  devices,  none  recommends  a 
conceptual  framework  that  may  be  used  by  purchasers  to  assess  the  safe  operation 

7  8 

of  any  device.  ’ 

This  paper  provides  an  overview  of  results  from  a  series  of  qualitative, 
retrospective  studies  that  examined  how,  and  to  what  extent,  patient  safety 
considerations  were  incorporated  into  the  device  purchasing  process  at  three 
hospital  sites  where  recent  intravenous  medication  infusion  pump  purchases  had 
been  made.  We  chose  to  examine  the  purchase  process  surrounding  infusion 
pumps  because  they  are  one  of  the  medical  devices  most  commonly  used  in 
hospitals,  and  they  are  well  represented  in  medical  device  error  reports.  The  three 
studies  were  created  using  a  conceptual  framework  that  considered  the  interactive 
role  of  patient  safety  in  the  device  purchasing  process;  the  quality  of  the  process 
as  it  relates  to  patient  safety;  and  the  patient  safety  attitudes  and  knowledge  of 
those  involved  with  or  affected  by  the  purchasing  decision.  First,  individuals 
involved  in  the  purchasing  decisions  were  questioned  using  a  structured  interview 
process,  to  better  understand  the  procurement  process.  Next,  end  users  were 
observed  as  they  operated  the  purchased  infusion  pumps  in  patient  care  settings, 
to  evaluate  the  means  by  which  they  addressed  safety  issues  that  may  have 
surfaced  after  the  pumps  were  put  into  practice.  Third,  focus  groups  comprised  of 
infusion  pump  users  were  conducted  at  each  site,  to  further  assess  safety  issues 
with  the  selected  devices.  The  latter  two  aspects  of  the  framework  provided 
additional  evidence  for  evaluating  the  quality  of  the  device  procurement  process. 
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Methods 

Participants 

Three  health  care  organizations  were  chosen  as  the  study  sites;  two  in  Houston 
and  one  in  the  New  York  Metropolitan  Area.  Twenty-nine  health  care  employees 
were  interviewed  in  all:  9  from  Site  1,11  from  Site  2,  and  9  from  Site  3.  All 
subjects  were  involved,  albeit  to  different  extents,  in  the  latest  purchase  of 
infusion  pumps  at  their  respective  sites.  The  participants  represented  different 
levels  of  the  hospital  hierarchy  and  different  employee  work  groups.  All  subjects 
at  Site  1  were  hospital  employees,  including  administrative,  management,  and 
technical  staff  members,  as  well  as  five  registered  nurses  (RNs)  who  were  device 
end  users.  Three  subjects  at  Site  2  were  members  of  the  health  care  system’s 
corporate  office  staff  and  eight  were  hospital  employees.  Of  the  eight  employees, 
four  were  end  users  of  the  infusion  pumps  (three  RNs  and  one  trauma  nurse),  and 
four  were  administrative  staff  including  the  biomedical  engineering  director  and 
the  materials  management  director.  The  sample  at  Site  3  included  one  biomedical 
engineer,  four  administrators,  one  physician,  and  three  nurse  managers.  The 
participants  were  selected  following  informal  discussions  at  each  site  to  ensure 
that  the  samples  were  representative  of  all  individuals  involved.  Each  interviewed 
person  also  was  asked  to  suggest  names  of  others  who  played  key  roles  in  the 
purchasing  process,  and  efforts  were  made  to  contact  and  interview  these 
additional  individuals.  The  key  personnel  from  all  sites  were  included  in  the 
study.  Approvals  were  obtained  from  the  Institutional  Review  Boards  at  each  site 
and  the  subjects  gave  their  written  informed  consent,  prior  to  the  interviews. 

While  assessing  the  results  presented  below,  bear  in  mind  that  while  patient 
safety  was  a  primary  concern  for  most  of  the  participants,  none  of  those 
interviewed  had  formal  training  in  human  factors  or  interface-related  device  usage 
safety.  Likewise,  none  of  the  study  sites  had  guidelines,  tools,  or  a  fonnal  process 
in  place  for  incorporating  safety  into  the  device  purchasing  process.  Some 
technical  staff,  such  as  biomedical  engineers,  were  trained  and  highly  skilled  at 
evaluating  the  technical  safety  of  the  devices.  Their  training,  however,  did  not 
include  formal  techniques  for  user  interface  evaluation. 

Interview  instrument 

The  interview  instrument  was  based  on  a  conceptual  framework  of  the 
purchasing  process.9  The  conceptual  framework  captured  the  aspects  of  interest  in 
the  study  domain,  including  key  factors,  variables,  and  the  relationships  among 
them.  This  study  addresses  three  key  factors:  the  role  of  patient  safety  in  the 
device  procurement  process;  the  efficacy  of  the  procurement  process  in 
addressing  patient  safety  concerns;  and  the  attitudes  and  knowledge  that 
participants  in  the  process  had  toward  patient  safety.  These  three  factors  were 
found  to  interact  in  various  ways  and  influenced  one  another. 

A  list  of  open-ended  interview  questions  was  developed  using  the  conceptual 
framework.  These  questions  focused  on  the  events  that  triggered  the  purchasing 
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process,  the  steps  identified  in  the  purchasing  process;  the  means  by  which  the 
safety  of  the  infusion  pumps  was  evaluated  and  incorporated  into  the  selection 
process;  the  patient  safety  knowledge  and  attitudes  of  the  participants;  and  the 
perceived  decision  and  process  quality.  The  interview  instrument  contained  eight 
sections:  (1)  introduction;  (2)  purchase  process;  (3)  implementation  of  the 
decision;  (4)  results  of  the  decision;  (5)  quality  of  the  safety  evaluation  and 
decisionmaking  process;  (6)  device  user  questionnaire  (given  only  to  device 
users);  (7)  safety  attitudes  and  knowledge;  and  (8)  demographics.  The 
introduction  contained  eight  questions  designed  to  examine  the  subject’s  work 
assignment,  years  of  experience,  and  whether  or  not  they  operate  the  infusion 
pump  in  the  course  of  their  work.  The  purchasing  process  section  was  comprised 
of  15  questions.  The  first  three  questions  examined  the  subject’s  role  in  the 
process,  their  past  experience  with  purchase  decisions,  and  their  knowledge  of  the 
factors  that  triggered  the  purchase  process.  The  fourth  question  asked  the  subject 
to  describe  the  purchase  process  from  start  to  finish.  This  was  followed  by  1 1 
additional  questions  designed  to  explore  the  details  of  the  purchase  process, 
including  issues  such  as  the  selection  of  alternative  devices,  how  the  purchasing 
group  was  fonned,  what  factors  were  used  to  evaluate  devices,  how  these  were 
combined,  who  made  the  final  decision,  and  what  role  patient  safety  played  in  the 
decision.  The  third  section  contained  two  questions  that  examined  the 
implementation  of  the  decision.  The  first  of  these  questions,  asked  of  all  subjects, 
examined  how  the  device  was  introduced  and  implemented  within  the  hospital. 
The  second  question,  asked  only  of  device  users,  looked  at  employee  feedback 
and  whether  it  was  encouraged  during  the  device  roll-out  process.  The  fourth 
section,  results,  contained  five  questions  intended  to  examine  safety  issues 
discovered  after  the  infusion  pump  implementation,  whether  user  feedback  had 
been  encouraged,  and  if  the  feedback  had  resulted  in  any  changes.  The  fifth 
section  contained  seven  questions  that  solicited  the  subject’s  opinions  regarding 
the  quality  of  the  pump  safety  evaluations  and  the  purchase  process.  This  section 
included  questions  that  asked  the  subject  to  list  what  they  considered  to  be  the 
positive  and  negative  aspects  of  the  process;  whether  they  felt  the  best  purchase 
decision  was  made  with  respect  to  patient  safety;  and  solicited  suggestions  for 
improving  the  selection  and  purchase  process.  Section  6  was  a  questionnaire 
given  only  to  users  of  the  infusion  pumps.  It  was  intended  to  measure  their 
satisfaction  with  the  selected  device.  Section  7  surveyed  the  subject’s  attitudes 
toward  errors  and  error  prevention.  It  involved  a  medical  error  scenario  and 
several  followup  questions  regarding  factors  that  may  have  contributed  to  the 
error  and  actions  that  might  be  taken  to  prevent  a  recurrence.  Section  8  was  used 
to  collect  additional  demographic  information,  including  age,  education,  and 
background  training. 

Interviews 

Each  subject  was  interviewed  after  written  informed  consent  had  been 
obtained.  At  Sites  1  and  2,  two  individuals  took  notes  during  the  interviews,  typed 
them  into  a  word  processor,  compared  the  transcripts  and  combined  them 
immediately  following  the  interview.  One  of  the  interviewers  at  Site  3  audio  taped 
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eight  of  the  nine  subjects.  One  of  the  subjects  asked  not  to  be  audio  taped,  so  that 
individual’s  responses  were  instead  recorded  by  hand.  The  identities  of  the 
subjects  were  removed  immediately  from  all  notes  and  transcribed  data. 

Data  analysis 

A  coding  scheme  based  on  the  detailed  questions  in  the  interview  instrument 
was  developed  to  summarize  the  subjects’  responses  to  the  interview  questions. 
Researchers  used  the  coding  scheme  to  retrieve  and  organize  the  interview  data. 
The  scheme  included  codes  for  the  following  components:  demographics; 
triggers;  purchasing  process;  safety  evaluation;  participants’  knowledge  of  device 
safety  or  human  factors;  quality  feedback;  organizational  information;  and 
suggestions.  Each  of  these  components  then  was  further  deconstructed,  using 
additional  coding.  For  example,  the  purchasing  process  includes  participants, 
purchasing  group  formation,  techniques/steps  used,  influencing  factors,  etc.  The 
code  “PR-FC,”  for  instance,  stands  for  influencing  factors  in  the  purchasing 
process.  A  list  of  detailed  code  descriptions  was  developed  for  use  by  the  coders. 
Each  set  of  interview  notes  was  coded,  after  which  the  researchers  reviewed  the 
handwritten  interview  notes  and  summarized  the  interview  on  a  contact  summary 
form,  as  a  means  of  developing  an  overall  summary  of  the  main  points  emerging 
from  each  contact. 

Once  all  the  interviews  at  a  given  site  were  completed  and  coded,  the 
researchers  reviewed  the  coded  data,  question-by-question,  in  an  effort  to  develop 
an  overall  picture  of  the  purchase  process  for  the  site.  The  main  objectives  of  this 
process  were  twofold:  an  examination  of  the  consistencies  and  inconsistencies 
among  the  views  of  the  subjects  at  a  given  site;  and  the  development  of  a  broader 
view  of  the  purchasing  process.  The  general  overview  of  the  purchasing  processes 
at  each  of  the  three  sites  is  reported  below. 


Results 

The  results  from  the  structured  interviews  were  categorized  according  to  the 
major  areas  of  interest:  factors  that  triggered  the  purchase  process;  the  purchase 
process  itself;  how  safety  was  evaluated  and  incorporated  into  the  purchase 
process;  and  the  perceived  decisionmaking  and  process  quality.  Additional  details 
of  the  purchasing  process  for  Site  3  have  been  published  elsewhere.10  Detailed 
analyses  of  the  purchase  processes  at  the  two  remaining  sites  are  in  preparation,  as 
is  a  detailed  analysis  of  the  safety  attitudes  of  the  participants  and  a  followup  study 
designed  to  examine  attitudes  toward  device  usage  safety  in  greater  detail. 1 '' 12 

Triggers 

Triggers  varied  across  the  sites.  At  Site  1,  a  surplus  of  equipment  funding  [for 
a  particular  intensive  care  unit  (ICU)]  prompted  the  ICU  director  to  ask  the  nurses 
on  the  unit  what  equipment  they  would  prefer.  The  nurses  requested  new  3- 
channel  infusion  pumps  to  replace  the  2-channel  pumps  they  had  been  using.  A 
business  merger  at  Site  2  (involving  a  large  health  care  system)  led  those  involved 
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with  the  purchasing  process  to  replace  their  existing  pumps  with  equipment 
specified  in  a  previously  negotiated,  systemwide  purchase  contract.  The  staff 
at  Site  2  felt  that  the  pump  selected  for  use  across  the  system’s  hospitals  did  not 
meet  Site  2’s  unique  clinical  needs.  Onsite  administrators  set  up  a  purchasing 
committee  to  evaluate  and  recommend  a  pump,  in  an  attempt  to  resolve  the 
conflict.  The  purchasing  process  at  Site  3  was  triggered  by  an  infusion  pump 
leasing  agreement  that  was  about  to  expire.  Although  the  triggers  varied  across 
the  sites,  it  should  be  noted  that  patient  safety  was  not,  in  itself,  a  trigger  for  any 
of  the  sites.  Participants  at  Site  2  had  named  safety  as  a  primary  issue  in  their 
objections  to  the  pump  previously  selected  for  purchase  and  use  within  the  health 
care  system.  In  particular,  they  felt  that  the  pump’s  physical  size  and  weight  and 
the  limited  number  of  channels  on  the  model  in  use  within  the  system,  combined 
with  the  clinical  complexity  of  their  patients,  would  lead  to  situations  in  which  too 
many  pumps  would  be  positioned  around  each  patient.  Such  a  scenario  could 
create  a  potential  safety  hazard  for  the  care  providers  as  well  as  the  patients. 

Purchasing  process 

With  respect  to  safety,  the  important  aspects  of  the  overall  process  include  the 
number  of  alternative  devices  considered,  those  who  participated  in  the 
purchasing  process,  and  the  general  process  leading  up  to  the  selection  of  one 
device  over  the  others.  Specific  device  safety  evaluations  and  their  use  in  the 
process  are  described  in  the  following  section. 

The  selection  of  alternative  devices  varied  by  site,  but  was  relatively  limited 
across  all  sites.  Only  one  model  of  infusion  pump  was  considered  for  purchase  at 
Site  1,  where  the  purchase  concerned  a  small  quantity  of  infusion  pumps  for  a 
single  ICU.  The  selected  3-channel  pump  was  already  in  use  in  another  unit  of  the 
same  hospital,  and  the  person  leading  the  purchasing  process  felt  that  the  hospital 
had  a  good  history  with  the  pump  manufacturer  and  its  pumps.  The  same 
individual  further  cited  a  corporate  contract  with  the  pump  manufacturer  and 
positive  feedback  from  the  nurses  who  were  already  using  the  3 -channel  pump.  A 
biomedical  engineer  at  the  same  site  stated  that  because  of  the  large  discount 
afforded  by  the  corporate  contract,  the  hospital  had  to  go  with  the  manufacturer  of 
the  selected  3 -channel  pump. 

The  purchasing  committees  at  Sites  2  and  3  considered  two  pump  models 
each.  The  choices  at  Site  2  consisted  of  the  pump  that  the  parent  health  care 
system  wanted  the  hospital  to  use,  and  another  manufacturer’s  multichannel  pump 
that  was  already  in  limited  use  at  the  same  hospital.  Staff  administrators  at  Site  3 
recalled  that  many  different  infusion  pumps  were  considered  early  in  the  process, 
however,  a  review  of  meeting  minutes  showed  that  only  two  devices  were  given 
serious  consideration.  According  to  the  minutes,  a  newer  version  of  the  pump  then 
in  use  at  the  hospital  was  first  considered.  After  entering  into  financial 
negotiations  and  conducting  clinical  evaluations,  the  purchasing  committee  found 
the  pump  had  significant  drawbacks  (e.g.,  heavy  weight  and  poor  display 
readability).  It  was  at  this  point  that  the  purchasing  committee  began  to  consider  a 
second  pump.  It  is  possible  that  the  committee  may  have  given  some 
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consideration  to  additional  pumps  very  early  in  the  process,  but  this  is  not 
reflected  in  the  meeting  minutes. 

The  ICU  director  and  the  nursing  manager  at  Site  1  were  the  only  staff 
directly  involved  in  the  purchasing  decision.  They  obtained  oral  feedback  from 
the  unit  nurses  prior  to  making  the  decision.  The  Chief  Financial  Officer  was 
involved  only  from  the  standpoint  of  approving  the  request  for  the  selected  pump. 
In  contrast,  a  variety  of  professionals  from  the  engineering,  administrative,  and 
clinical  areas  at  Sites  2  and  3  were  involved  in  the  purchasing  process.  Across  all 
three  sites,  however,  the  role  of  the  infusion  pump  end  users  was  limited  to 
providing  feedback  on  the  pumps  under  consideration — they  were  allowed  no 
direct  role  in  the  selection  of  alternative  devices  or  in  the  final  decision. 

Although  the  details  of  the  process  varied  from  site  to  site,  each  group 
followed  the  same  basic  format  of  evaluating  the  device  choices — which  included 
collecting  user  feedback — and  then  making  a  decision.  At  Site  1,  the  pump  was 
brought  in  for  a  2-week  trial,  after  which  the  ICU  director  and  nursing  manager 
collected  oral  feedback  from  those  who  had  used  the  device.  Nurses  at  Site  2  were 
given  a  survey  asking  their  opinions  on  the  important  features  of  infusion  pumps 
and  both  pumps  were  set  up  for  the  nurses  to  try.  The  pumps  were  not  put  to  use 
in  clinical  care  settings.  The  most  notable  aspect  of  the  process  at  Site  2  is  that  it 
appears  to  have  been  used,  in  large  part,  to  demonstrate  the  inferiority  of  the 
health  care  system’s  recommended  pump,  while  at  the  same  time  emphasizing  the 
ability  of  the  alternative  pump  to  better  meet  the  hospital’s  unique  clinical  needs. 
The  two  pumps  considered  for  purchase  at  Site  3  were  evaluated  in  clinical  trials 
that  included  the  use  of  vendor-supplied  surveys  to  collect  solicited  user 
feedback;  however,  a  different  survey  was  provided  for  each  of  the  pumps. 

Safety  and  cost  were  mentioned  as  major  factors  in  the  decisionmaking 
process  at  each  of  the  sites.  However,  except  for  a  small  number  of  device 
characteristics  (e.g.,  the  number  of  channels  and  the  physical  weight  and  size), 
safety  issues  were  limited  largely  to  technical  specifications  (i.e.,  the  ability  of  the 
device  to  function  properly  and  operate  within  the  standards  of  accuracy  stated  by 
the  manufacturer). 

Safety  evaluation  and  its  use  in  the  purchasing  process 

Although  technical  safety  was  an  obvious  consideration  during  the  purchasing 
processes,  factors  that  play  a  role  in  device  usage  errors  rarely  were  considered. 
Many  of  those  interviewed  seemed  largely  unaware  that  a  pump’s  interface  design 
could  induce  device  usage  errors.  Some  users  did  mention  “ease  of  use”  or  “ease 
of  programming,”  which  indicates  a  general  awareness  that  the  device  interface 
can  have  an  effect  on  safety.  The  number  of  channels  and  the  pump’s  physical 
size  and  weight — clearly  important  safety  factors — did  play  prominently  in  most 
of  the  participants’  views  of  the  decisions. 

As  noted  previously,  feedback  at  Site  1  was  limited  to  oral  comments  from 
nurses  who  had  used  the  one  model  of  pump  brought  in  for  the  trial.  Those 
involved  in  the  selection  process  at  Site  2  were  concerned  more  with  the  weight 
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and  the  number  of  infusion  channels,  and  less  with  the  device  interface.  To 
demonstrate  the  potential  for  safety  problems  with  the  (2-channel)  pump  already 
in  use  at  other  facilities  operated  by  the  same  health  care  system,  they  placed 
eight  in  a  room  and  used  photographs  to  demonstrate  their  concern  that  the  pumps 
would  become  a  workspace  obstruction  and  would  impede  access  to  the  patient. 
Likewise,  the  survey  conducted  at  Site  3  contained  four  questions  for  evaluating 
the  design  of  the  interface.  But  the  questions  addressed  only  four  features  of  the 
pump  and  did  not  pennit  an  adequate  evaluation  of  the  device  interface.  Thus,  the 
sites  collected  very  little  information  regarding  the  safety  of  the  various  pump 
user  interfaces,  apart  from  the  largely  informal  user  feedback  incorporated  into 
the  evaluation. 

Perceived  decision  and  process  quality 

Administrators  across  the  three  sites  considered  the  final  purchase  decisions  to 
be  correct  and  in  the  best  interests  of  patient  safety.  The  clinicians’  opinions  of 
the  decisions  varied  by  site.  Those  directly  involved  in  the  decisions  at  Sites  1  and 
2  felt  the  best  decision  was  made.  Some  clinicians  at  Site  3  expressed  the  opinion 
that  users  were  less  satisfied  with  the  selected  pump,  and  that  user  feedback  had 
not  been  made  a  priority  in  the  decisionmaking  process.  Several  device  users  at 
each  of  the  sites  said  that  their  opportunity  for  feedback  was  too  limited  and  that 
the  provided  feedback  was  largely  ignored.  Their  lack  of  direct  involvement  with 
the  purchase  process  left  these  users  hard-pressed  to  understand  the  basis  on 
which  the  purchase  decision  had  been  made,  and  they  said  they  had  difficulty 
determining  if  the  correct  device  choice  had  been  made.  A  technical  staff  member 
at  Site  1  said  during  the  interview,  “I  begged  and  pleaded  with  administration  not 
to  buy  the  pump,”  because  this  person  was  aware  of  several  FDA  recalls  and 
alerts  involving  the  selected  pump,  and  felt  that  the  older  model  was  easier  to  use. 

Perceived  quality  of  process  varied  by  site 

Administrators  at  most  of  the  sites  were  generally  pleased  with  the  process 
and  their  role  in  it.  The  lone  exception  was  Site  2,  where  the  hospital 
administrators  said  they  felt  the  process  would  have  been  more  efficient  if  the 
administrators  at  the  health  care  system  level  would  have  collaborated  with  them. 
Administrators  with  the  parent  health  care  system  said  that  they,  too,  could  have 
made  the  process  more  efficient  and  cost-effective,  if  only  the  hospital  purchasing 
committee  had  been  more  cooperative.  While  the  device  users  across  all  sites  had 
little  insight  into  the  purchasing  process,  those  end  users  at  Sites  2  and  3  said  they 
felt  that  not  enough  weight  had  been  given  to  their  feedback.  The  concern  they 
had  with  “not  being  heard”  was  voiced  by  the  users  at  every  level  of  the 
decisionmaking  process.  The  device  users’  opinion  of  the  process  quality  at  Site  1 
varied.  Half  of  the  users  said  that  they  had  played  a  valid  role,  while  the  other  half 
said  they  were  given  little  or  no  opportunity  for  input — or  that  their  input  was  not 
a  factor  in  the  decisionmaking  process.  Likewise,  the  technical  staff  member 
interviewed  at  Site  1  expressed  the  opinion  that  the  process  had  been  flawed 
because  the  biomedical  engineering  department  staff  had  the  most  knowledge  of 
the  pumps,  but  did  not  play  a  direct  role  in  the  decisionmaking  process. 
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Discussion 

Human  factors  research  has  shown  the  importance  of  involving  all 
stakeholders  in  medical  device  and  software  evaluations.  ~  Additional  evidence 
also  has  shown  that  the  successful  implementation  of  new  technology  depends,  in 
part,  on  the  early  involvement  of  all  stakeholders. 14  With  respect  to  purchasing, 
this  research  makes  a  case  for  involving  a  wide  range  of  professionals  in  the 
process — including  anyone  whose  responsibilities  are  affected,  directly  or 
indirectly,  by  the  device  purchasing  decision  or  the  clinical  use  of  the  device.  For 
infusion  pumps  this  includes,  at  a  minimum,  nurses  (who  program  the  pumps), 
physicians  (who  write  the  orders),  pharmacists  (who  fill  the  orders),  biomedical 
engineers  (who  repair  the  devices),  quality  improvement  staff,  unit  managers 
(who  supervise  the  nurses),  and  patients,  as  well  as  those  who  train  the  pump 
users,  and  the  administrators  who  must  account  for  the  cost  of  the  pumps  and  their 
maintenance. 

The  results  of  this  study  reveal  strengths  and  weaknesses  in  the  use  of  patient 
safety  concerns  as  a  factor  in  the  device  purchasing  process.  On  a  positive  note, 
participants  at  all  sites  regarded  patient  safety  as  an  important  issue,  and  all  used 
some  type  of  user  feedback  mechanism  to  assess  user  satisfaction  and  to  express 
safety-related  concerns.  In  addition,  Sites  2  and  3  included  a  broad  range  of 
stakeholders  in  the  purchase  process.  That  Site  1  did  not  do  this  is  likely  a 
reflection  of  the  facility’s  smaller  size,  and  the  modest  number  of  pumps  that  were 
being  purchased  for  a  single  ICU.  The  variance  is  worth  studying,  however,  as  it 
could  reflect  a  difference  in  the  decisionmaking  processes  at  smaller  and  larger 
hospitals,  or  a  difference  in  the  methods  by  which  smaller  and  larger  purchases 
are  decided. 

The  results  also  reveal  a  number  of  weaknesses  in  the  current  purchasing 
processes  at  the  three  studied  sites.  The  consideration  given  to  alternative  devices 
was  limited  across  all  sites,  as  participants  at  one  site  considered  only  a  single 
infusion  pump  for  purchase,  and  the  groups  involved  in  the  process  at  the 
remaining  two  sites  considered  two  pump  models  each.  Purchase  decisions  should 
ideally  involve  a  broad  range  of  available  devices,  to  ensure  the  safest  device  is 
selected.  At  each  of  sites  where  two  different  models  of  pumps  were  debated, 
consideration  was  given  first  to  a  single  device  and  then,  after  that  device  showed 
problems,  to  a  second  device. 

The  study  further  revealed  potential  problems  with  the  composition  of  the 
purchasing  groups.  End  users  have  the  greatest  knowledge  of  a  device’s  strengths 
and  weaknesses,  and  of  the  environment  and  tasks  to  be  performed  with  the 
device,  yet  none  of  the  three  purchasing  teams  included  end  users.  Likewise, 
patients  were  not  included  on  the  purchasing  teams. 

While  user  feedback  was  solicited  and  collected  at  each  of  the  sites,  most  of 
the  end  users  said  they  were  given  only  a  limited  opportunity  to  contribute 
feedback.  Moreover,  they  expressed  frustration  at  the  lack  of  weight  given  to  their 
feedback  in  the  pump  selection  process.  In  addition,  none  of  the  sites  used  formal 
methods  for  assessing  safety  issues  that  may  arise  from  the  design  of  the  device 
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interface.  Instead,  much  of  the  safety  evaluation  focused  on  the  technical  safety 
specifications  of  the  pumps  and  informal  user  feedback. 

Another  primary  process  limitation  at  all  three  sites  involved  assertions  that  a 
particular  make  and  model  of  infusion  pump  was  identified  as  the  preferred 
product,  after  which  the  discussion  and  selection  process  was  carried  out 
primarily  to  justify  the  choice  of  that  product.  For  example,  the  lead 
decisionmakers  at  Site  2  decided  early  in  the  process  that  the  2-channel  pump 
being  recommended  for  purchase  by  the  parent  health  care  system  was  not  as  well 
suited  to  the  hospital’s  needs  as  the  3-channel  version  of  the  pump  the  hospital 
had  been  using.  The  purchasing  committee’s  subsequent  work  appears  to  have 
been  done  solely  to  demonstrate  the  superiority  of  the  preferred  pump,  and  the 
inferiority  of  the  pump  recommended  by  the  health  care  system.  A  selection 
process  that  is  undertaken  to  justify  a  prior  decision  is  especially  problematic, 
because  it  indicates  the  likelihood  of  a  confirmation  bias  (i.e.,  a  bias  used  to 
solicit  data  that  confirms  a  particular  hypothesis),  while  further  suggesting  that 
information  supporting  alternative  hypotheses  may  have  been  discounted  or 
ignored.15  Studies  of  the  confirmation  bias  have  shown  that  it  often  leads 
individuals  to  believe  in  a  faulty  hypothesis.16  Thus,  individuals  on  these 
purchasing  committees  may  have  believed  that  they  selected  the  best  available 
device,  when  in  fact  they  were  not  given  or  chose  to  ignore  data  that  may  have 
discounted  their  choices. 

Patient  safety  was  named  a  major  factor  at  each  of  the  sites,  however,  we 
found  that  the  purchasing  groups  gave  more  consideration  and  emphasis  to 
technical  safety  (i.e.,  whether  the  pump  was  operating  within  certain  technical 
parameters).  There  was  relatively  little  awareness  among  the  groups  of  the  device 
interface  role  and  how  it  might  directly  impact  patient  safety  by  increasing  (or 
decreasing)  the  incidence  of  device  usage  errors.  As  a  result,  purchasers  showed  a 
tendency  to  judge  infusion  pump  software  features  (e.g.,  drug  dosage  calculators) 
as  being  equal  with  respect  to  their  influence  on  safety,  whereas  human  factors 
engineering  has  shown  that  the  effectiveness  of  such  features  is  largely  a  function 
of  their  user  interface.  In  contrast,  pump  weight  and  the  number  of  pumps  needed 
per  patient  played  a  major  role  in  the  purchase  decisions,  and  some  users  were 
clearly  concerned  with  “ease  of  use”  and  “ease  of  programming.”  This  indicated 
some  awareness  of  the  human  factors  issues  involved  with  device  use  safety. 
Overall,  however,  these  results  reveal  inconsistencies  with  respect  to  the 
participants’  understanding  of  patient  safety  and  its  role  in  the  decisionmaking 
process,  as  well  as  the  extent  to  which  safety  was  a  factor  in  the  infusion  pump 
procurement  decisions. 

Another  perplexing  dimension  of  the  purchasing  process  involved  the  survey 
participants’  collective  concern  for  patient  safety,  and  their  inability  to  define  or 
articulate  those  factors  with  a  direct  connection  to  patient  safety.  All  members  of 
the  subject  groups  felt  that  patient  safety  was  an  important  dimension  and  one  that 
should  be  addressed  in  the  overall  process,  however,  they  were  unable  to  agree 
upon  a  conceptual  model  with  which  they  could  identify  and  order  the  aspects  of 
infusion  care  most  important  to  a  successful  patient  outcome. 
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Although  most  of  those  involved  directly  with  the  purchases  felt  that  the  best 
decisions  had  been  made  and  that  device  safety  played  a  primary  role  in  the 
decisions,  the  device  users  were  less  certain  of  the  decisions.  As  noted  above, 
some  participants  directly  involved  in  the  purchase  decisions  expressed  concerns 
that  the  pump  selected  for  purchase  was  not  necessarily  the  safest  model 
available.  Although  our  interview  results  showed  that  the  perceived  quality  of  the 
decisionmaking  process  was  mixed,  our  independent  analysis — done  from  a 
human  factors  engineering  perspective — revealed  numerous  process  limitations. 
The  limitations  identified  in  this  study  do  not  reflect  a  lack  of  concern  for  patient 
safety  among  the  participants.  All  participants  in  this  study  expressed  the  opinion 
that  patient  safety  was  a  major  dimension  of  the  purchasing  process  and  they 
attempted  to  maximize  patient  safety  throughout  the  process.  Rather,  the 
weaknesses  are  a  reflection  of  the  participants’  lack  of  training  in  device  safety, 
its  integration  with  the  purchasing  process;  and  the  need  for  formal  tools  or 
guidelines  to  advance  such  a  synthesis. 


Conclusion 

Despite  the  general  awareness  of  patient  safety,  its  importance,  and  its  role  in 
medical  device  comparisons  and  purchasing  decisions,  this  study  revealed  some 
serious  limitations  in  the  selection  process  at  the  three  sites.  It  also  calls  into 
question  the  methods  by  which  device  usage  is  used  to  generate  feedback  for  the 
selection  process.  These  results  illustrate  the  need  for  a  more  formal  method  of 
incorporating  patient  safety  considerations  into  the  medical  device  purchasing 
process.  We  presently  have  a  set  of  guidelines  in  development  to  help  hospitals 
better  emphasize  patient  safety  in  their  device  purchasing  decisions.  These 
guidelines  include  recommendations  on  those  types  of  personnel  to  be  included  in 
the  selection  group;  the  overall  process  of  device  evaluation,  tools  and  techniques 
for  assessing  device  interface  safety;  and  procedures  for  bringing  other  relevant 
factors  into  the  selection  and  purchasing  process. 
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